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Abstract 



Two experiments exfmiried /children 1 s ability to apply three different;; standards 
for evaluating their understandihg* Fi ve- , seven-, nine-, and eleven-year- . ' 
old children, were presented ^ith short narrative passages within which were 
embedded three types of problems (nonsense words, internal inconsistencies,* 
and "prior knowledge violations) , each of which could only be identified if a , 
specific standard of evaluation w^re useci ( lexical y internal consistency, and 
^external consistency, respectively) . - Since the* focus of the study was on the 
effectiveness with which children could apply the standards, rather than on 
the 1 ikeli hood that^they would spontaneously adopt ajjd then apply them, the 
subjects -were explicitly instructed i|£ advance that ythei r task was to find the 
"ml stakes, Moreover, the subjects ^e re giv'en immediate feedback §fter each 
trial and a second opportunity to find any missed problems. Although older 
children used all three^standards more effectively than younger children, 
overall problem ideritfe-cation was considerably better than that reported in 
non-1 nstrycted settings. The internal consistency standard was applied least 
effectively,* but even the youngest children were able to use it. The results 
illustrate the nped to consider comprehension monitoring' skill s with respect 
to specific standards of evaluation, rather than as a unitary phenomenon, v- 



Mill ti p] e .St a nd a ril s \ 



Children's Effective Use of Multiple Standards 
•for Evaluating Their Comprehension 



, Because comprehension monitoring is an ability Which has important educa- 
tional implications, the degree to which children monitor thei r comprehension* 

has become an issue of concern to many investigators* ^Educators ha ? '] on<j 

. " * = : ....*'" = * - ' • i* ■ ■ t . • .* * . ■ 

argued that- students should k§ep track of their understanding and should V 
" respond apprppM ately if they detect a failure to Understand (e.g., Huey, 
, 1906; Thorndike, 1917). However, more recently, empirical studies have 
demonstrated th &t children are surprisingly unlikely to carry out these 
eval ua£ i on 'and regul at i on act i vi t i es-(-e . g . , Garner, 1980; Mafkman, 1977, 1979; 
* Paris ^ Myers, 1981). While the research has shown that older and better 
students :;moni tor their comprehension more effectively than younger and poorer 
students, the latter group still shows considerable room for improvement, 
, The basic paradigm used in studies of comprehension monitoring is to 

• introduce a problem or "error 14 of some sort into a prose passage and assess 

* ; • ' the subjects 1 ability to detect it. The rationale for this procedure is that 
•subjects shoul d noti ce the probl ems if they are keeping a careful' check on % . 
* ^their understanding. However, the fact that subjects typically are -not ' 
, * .informed in advance that the passages are problematic can lead to serious 

.u^dir^stimatioh Icif their comprehension monitoring activities; This is because 
people tend to believe that the communications they receive will bf true, 
' cdmpleteV and informative (Grice, 1975) and consequently, they\ttempt to make 
sense^of any input ,/however confusing (&f, Baker, 1979) . , The few stud leg 

* which haye' alerted subjects tcPthfe presence af .probl ems generally report 
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higher levels of performance (eig. , h ii# M?. nan & Gprih, 1981 

- The nature of the. embedded prpbl .1 mav also nc^e „ strong influence on 
children's success pr failure on the . ^ r -, ners havfe used a number of 

different kinds of probl ems, such t ^» st we ds, violations of prior 
knowledge, and inconsistencies with 1 - th *xt itself • detection of these 
different problem .types requires the ..doi ic ion of different criteria or 

. • • : ■ ■ I * ^ =t' - .-.-' f 

standards of evaluation. In order to detect a nonsense word', a child must 
evaluate her understanding of individual, word meanings* In order to detect- a 
prior knowledge violation^ a child must consider how" the .ideas in the text 
'relate to what she al ready knows. And to detect an internal inconsistency, 
the child needs to evaluate the consistency of the ideas expressed, in the 
passage, Thus, if a child fails to adopt a particular standard of evaluation, 
she -will not notice the corresponding problems, This does not mean, however, 
that she did not evaluate her understanding along other dimensions* Since 
most studies have used *t)nly one type of embedded problem, they may present a 
misleading picture of children's comprehension monitoring skills;, ■ . 

There is some evidence that children are more likely to spontaneously 
adopt some standards thaft others. For example. Garner' (1981) found that 
children were more likely to use a lexical standard than an internal - 
consistency* stanbard, as indexed by their superior identification of difficult 
vocabulary items and their lack of 'Identification of any inconsistencies. And 
there • Is- also evidence that chi 1 dren adjust thei r ; standards accord i ng to the . 
instructions they receive. Markman and Gorin (1981) found that children who. 
were set to eval uate for' internal consi stency identi fled" more inconsistencies 
than falsehoods, while children set to«eva1 uate for external consistency 
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identified more falsehoods than inconsistences'. However, we do not Know V 
■ whether children can keep several different stances in n,nd ,nd use all o£- 
the " Sft 0^ A^Hionan y , „e have no>formation regards \. ; ^ 
develop»,ental changes in the variety of stendards' children can die; 

The present study was therofore'designed'to examine the effectiveness with 
which children of dl.ff.reot ages can apply multiple standards for evaluating 
their understanding. -Tnree specific standards were the focus of the ■ 
^^i^C^ifuU i«irt»Vi«««^; and'external consistency: > 
: .Effective u... of tS.se 'standards 'was operational ized ;as success at- finding- 
embedded npdsense words,, inconsistencies, and prior' knowledge violati™* ... 
respectively. . Ihc chiTW^r. specifically infold that they woulheed to' 
use these standards in ordef to: find the , Mistakes" in several passages. The ; 
• instructions were «ade. as elicit >s possible becaus^ the concern was with 
children's ability to us'e the Isfandards,, not wVth whether they would'" ■ 
spontaneously ad 6pt thee,. Moreover, standard use was fostnred^uring the • V 
experlniental «i,1bn: by providing 'chi ldren^ith tarndiate feedback and a " ■ 

^econd opportunity to identify missed problem. Also of interest', in addition 
to problem identification, : „ere ; the children's evaluative comments about other 
parts of the passages.' These comments; are important because they car, reveal V 
whether or not a particular standard-was in fact present in a given child's 
repertoire, and if so. whether *it was ^psed selectively. 

iwo experiments &r. carried out.' Inthe first, five-, seven-, and nine- 
year-old chiidren listened to the passa ges^hich jyere read aloud by the . 
exp-errmehter: In the second, eleven-year-bid children read the passages on 
thetr own and Jere later. Interviewed about their perceptions of standard 1 

^■^; / V:^;;^./;;^;:..■v• v', : ,v \; • ; 
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difficulty (p^gV.i Which type of* problem was ealiest to notice and why?), 1 It 
was expected that the older chiTdreruwould apply al l three types , of standards % 
more effectively than younger children, fV Wne with previous findings.^ " .-■ 
However, overal 1 performance levels were expected to be higher than- previously 
reported because of the specificity of th f e instructions and the provision of 

v : .= r ..:^ :t ■ - ■ ' : . ■-. ,:"« ■■ ^ ' . ■: ■ 1 * - 

feedback wiith a second ^chance to^find the problems' It was also expected 
that the lexical standard would be u§ed most effectively. and the internal 
consistency standard 1 east effectively, an expectation based on the fact th&t 
these two standards differ considerably -jn the level of processing required 
for their application* s ■ v r " 

i" ' . » : "8 : . * 

* < * / Experimental 

Method .-V- '•' ' ' ; -;-.r v - , ; • ' > , : , r ' - - 

Subjects , A total of B3 5-» 7-, and v9-^ar-old chi Idren participated in 
the study* The two older groups of children 'were enrolled in the first and 
third grades of a suburban public school ,. Thei~e were ,16 seven year olds (X 
age » 6 years, 11 months; nine boys) and 14 nine year olds (X age * 9 years, 0 
months; seven boys) - The youngest, group 'was drawn from two different suburban 
preschool s. There were 1 1 children from one preschool (X age =4 years, 8 
monthsj five boys) and 12 children from the second preschool (1C age = ' 5 years, 
.1 .month r -6 -boys)--.- All children .were tested during the last two Weeks" of the 
school year except for the children in the first preschool, who were seen in 
January,^ '. ■! 

-Material s , .\ The materials consisted_o_f_s_ijc__short nanrat-i ve passage 



dealing with topics and situations .familiar to young .children. Each passage 
v?a.s 7 tp 9 sentences in lenyttr (X = 8.1/) and contained from 69 to 82 words (X 
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; _ • . v • - : % - ,-, - ■ . ,. : .> - ■ . 

■•=- 75;. Each passage was modified to contain two of three different types of 

problems which tould be detected by us*, qf the appropriate Standard of^ 

evaluations knowledge violations, and internal 

inconsistencies* The nonsense words were selected from a list of two-syllabi 

"paralbgs , \ (Noble, 1952) and followed standard rules of English orthography, 

The nonsense words always replaced nouns that occupied the final position in 

-sentence, An" example of a target sentence containing a nonsense word is, 

"Mrs* Johnson cooked the pancakes -in a bladmer." The prior knowledge ■.-<-. 

violations were created by introducing information that conflicted with world 

knowledge that children were assumed to possess. An example sentence is 

ft ; » * ' . 

"Jack always used a. baseball bat to chop the wood." The internal i nconsis- 
tencies Involved two target sentences each, wh1 t ch were separated in .the 
passage by one Intervening sentence. The information contained 'In the two 
target sentences was contradictory. * An example is, "He [a rabbit] had dark 
brown fur that was as- soft as could be. He Was very fluffy and had a a • 
beautiful tail. A1V the other rabbits wished they had his snow white fur." ' 
Tw< examples of representative passages appear in Table 1. 

The problem types were systematical ly distributed throughout the passages 
such- that each- problem type appeaTed-wlttrievei y o ther proble rrrtyprg' twrc%7 oner 
in first position and once In second position. The placement of the problems 
within the passages was restricted in two ways: No problems appeared in 
either the first orjast sentence of any passage and when one of the problems 
was an -internal i nconsistency , the second probTem was-never-pTaced in the 
intervening sentence. • , ,^ ■ 



Multiple* Standards 



Insert Table 1 about here 



The experimental materials were developed from a larger set of 12 passages 

that were presented to 28 undergraduates drawn from an introductory psychology 

■ •. . - . . . , . ■ * : . • • . ^ . • • 

subject pool • . In order to verify that the problems would be perceived as such 

by adults, the students were instructed to read the passages carefully in 

search of the three different types of problems, They were to underline the 

problems when they found them and write alongside the target sentence the type 

of problem they considered it to.be. The response sheets were scored for the 

percentage of students correctly identifying eath problem- The detection rate 

. ■ ■ - » . '*■*". 

was consistently high, ranging from 82% %o 100%, with an overall mean of 94%* 

#• • • * * t .■■ . =• , *„ ■ 

The difference In detection rates among the three types of problems was 
nonsignificant* (96%, 95%, and 93% for nonsense words ^ prior knowledge 
violations, and i neons i s€ertjci es , respectively)/ A criterion of 93% detection 
wjas established as the cutoff for inclusion of the problem in the materials 
for the childrefu ' (In other t words , no more than two of the 28 subjects could 
miss the problem.) A second criterion was ^hat no more than two students 
_cbuld__cQmment that a nonpriblematic sentence in fact had some sort ot>-p^bljmu_ 
Taking these criteria Into account, six of the passages were selected any 
modified for use as experimental materials. 

The passages were arranged such that each type of problem appeared 
sequ en ti ally eithe r ~~w i t h in o r~ between stbrTesV^ "The "order" of 6 r e s entatToTT of ~ 
the passages was counterbalanced^ such that the first three passages for half 
of the subjects were, the last three passages for the other subjects, and vice 
versa. V • . % •. ■ r . ; . ' : 
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/ .Procedure. The children were seen individually during school ho»>s In a 
quiet room at their school . They wore told that they would be "listening to 
some stories that had mistakes 1 n ,them and tha^these mistakes were there '.; 
' because the writers had not been careful enough when they wrote the stories. 
'The children's job was to find the mistakes and -tell the experimenter. The 
three types of problems or "mi stakes" were described as follows.- words that 
ar^t really wrds; things. that^don'i real ly happen ttie^ it ia^s in the 
stbry; and things in one part of the stor^ that don't go with things in 
another part. Then a sample story containing one of each problem type was 
read and the children were asked to try to find the problems. Regardless of . 
their performance, ' the experimenter then cited the embedded 'problems as 
concrete examples of the tWee different types that , should' be sought,* The 
; ? h11dren we ^ not "told how many problems would be present in any given 
passage. J 

Since the primary purpose of the study was to assess ■children' s evaluatton 
skills under optimal-circumstances, the task was structured to provide an' 
opportunity for improvement in the child's performance within the session. To 
this end. the chiWren were given two chances to find the problems. Each 
story was read al oud once by the experimenter and the child., were asked to 



report whatever problems they noticed. >They were encouraged to report 
anything that "didn't seem quite right" even if they were not sure it was ' a ; 
mistake. If the children did not report both' problems that were present, they 
were told they had missed something and were asked to listen to the story 
again, To alleviate possible memory difficulties; they were encouraged to 
interrupt as soon as they noticed a problem. After the second reading, if the 
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^ children still d1"d"-not -"pep^ them, 
-ThTs explicit feedback in essence, served as additional instruction \L\n* using 
the standards off valuation. * \ ; : ■ ■ .-; / > \ : ■." * . 4 '' 
' A written record was kept of the children' s ingoing responses anai the / V 
sessions were also tape recorded, for later transcription-; The sessions range 
in length from 10 t'o 25 minutes^ - v 

Scoring. The response protocols were scored for; two different dependent 
measures: problem identification and standard application. Each:' problem was 
scored* as correctly identified or riot. If the children- were not specific 
enough In their initial responses to permit scoring, the experimenter 
"requested clarification. For example, if a child simply said the mistake "was 
"the part about snow white fur," the experimenter asked, "What about snow 
white' fur was wrong?" If the child then explained something to the effect ' 
that the story earlier said the rabbit had. brown fur, the problem was scored 
( , as identified. If the child indicated that some otherr component was 
problematic (e.g., "Them's no such thing as snow white fur"),' the problem Was 
scored as not identified. (Such comments, were classified, however,' as to "the 
-type of standard they repealed, as described below.) Responses were scored 

immediately,' since the next step in the procedure was dependent. on the 
Judgment (e.g. , if a chl 1 d was correct oh Tri al 1 .... Trl aj 2 w oul d not be 
necessary); However, all decisions were al so" checked ^Tter^he-tapeWhM, bgen 
transcribed >^nd an independent judge validated the decisions with 98%, 
agreement, '" ■ " /. - & ■ 

ATI scoring for the standard application^^ after the 

tapes were transcribed, Each response the chfTd rpde to any part of the story 



was classified as to whether it revealed the use of a particular standard of 

' . . - • i • 

evaluation. The classification scheme consisted of the three target ' ] '\S-£ Z 
■ ■ •" • . ■' : ' • ' ■' . ' " ' ■'' ' ' ' ' : . •■ f- '*. ' *.'•• ' *. 

standards, lexical, external consistency and internal consistency, as well as . 

r ' : £v£lT-.. ' ' -' ;;v •■■ iF'^i- I . V ; ■ -'^-yZU " .: v.:. 7 i 7 - : 
four others: propositional cohesiveness, structural cohesi yeness ,,• J.y : ^^^M^t 

"necessary to eStabl ish a categ^ny -for "other" respqnses , because many of the ; . ? t A 

>" - • ;.-^>v : ' ' • - •-• •< , \y i-'.--;;.."- 1 --::.: ■ .-i--. -i".----.""-.-";.- •- -•. •' •-• -- ,r - .7 

'five olds made norieval ua^y^ comments (fe.g^ v^'We .have . a dog too") The • ■ 

v''. ' ' ' ; ' • '• ' .' . ; . *•,;•:; •: '-; ' ' ' • 

protocols 
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standards, the focus here witl be on the target standards -only.- 

..-;.v' r .-- ''.V'.. ; -.7': ' ' ' ' 7; . 7" -7;. '•;> . / => 

Results' and Discussion ' ' 




; :th^ 

section will focUs on chiT 

the second ;^11J'v.e^a^ii'1'ne '."thie ; sjt-ahd^^s' v*cbiTdr!B_K'.' : u's*d ^when. 1 dent f fy 1 ng ' ajw : .'baH:s- : 
of the pissages aS pro 

exterj^to which children's use of, the three standards is similar to adult • 
usage and the second section examines standard use from the individual: child's 
perspective of what is or is not problematic. _ ; 7 

P rob! em i dentif i cat i on . Separate analyses were carried out for the number 
of problems the children identified- after. hearing t^ 

for the ' total" number /of.p.robl^^ldentffl^d^ In addition, a .thi rd analysis: 
included trial as a factor in order to examine* differential improvement when 
given a second opportunity to find the problemms. In this latter analysis, .1 f 
-. a problem was. Identified on Trial 1 V it was also scored as Identified on Trial; 
7 2, even though the child had not actually been asked to identify it; 3 



- 1^ problemsYof each ]t^f^ that tKe v 

-xhiidreri Wentif after a t 

second, listening as necessary .(Trial 2)J Two 3 (age) x 3 (problem type) mixed 
analyses of variance were carried out using thise two^data setsi With ageias i 
between- subjects factor, and problem type as a within-subjects fartor» Y ^ 

v ? v k Insert Table 2 about here \ ; :■'■':>■'"■ 

Consider first the results for the f i rst trial , Reliable mai n £ effects of 
age and problem type were obtained, and the interaction of the two factors Was 
also reliable, _As^ expected, older children were more successful at 
identifying al 1 types of problems than younger children , F(2,50) = 53.70, "p < 
.001 • The a year olds identified an average of 3.26 problems of each type, 
the 7 year o]ds 2.16 and the 5 year olds 1.19. AIT comparisons between means 
were si gnifi cant (Fisher' s Isd - .39, _£ < .05). Also as expected, internal 
inconsistencies were least likely to be identified (£(2,100) ^ 10.92, £ < 
- 001 - Slibjects identi fi ed an average of 1-66 i nconsistencies , 2.06 nonser>se 
words , and 2.38 prior knowledge violations. All of the differences between 
means were reliable (Fisher's Isd = .28, £ < .05) . The interpretation of the 
effect of problem type is mediated by the Interaction with age, £(4,100) = 
2.^8, £ < .05, Whereas the 7- and 9-year-old children had comparable 
identification of nonsense words and prior knowledge violations, with poorer 
Identification oV inconsistencies, the 5 year olds had comparable : -.V' 
identification of nonsense words and inconsistencies, with superior 
identtf i cation of prioc know! edge violations (Fisher's Tsd ? ,50V £ < .05). • 



• W differences iri; : 

problem i dent 1 f Icati on were not constant across probl em * types^ >low^ 

: - - • ' ' ' • •,"""•*« . • — i\~ " v : •* - • . ' *" . - ! ' * ■■■■■ ■ . " = = ? s '- .«•.*••.•." •'. ■*.•.".".*■■-•••"'•■• iV :'■ *'". •' . - 

.;. -.';,'•_.*' . " , = "V " •"- 1 " ' "-=-■ " r ''-~- ■■ V - ~" ' " '. ' • " " - ": _ ~' '■ : • . ' • .. : . "j -'■•=."':."••'.'"•". V A, • 

^pattern dlffertrtces; propped ou by the second trial , as r revealed i n :the second 
anal vsa^ of van" an ce . Aga i nV the jfiria i n effects of age' and probTem type Were 

" r pel it'a^^^lkjpt' "thi" "-'-'I nte r^dct 1 on - " wbs - ^riot ("F C2^'"S0") 36'. 26 -,--V " ■"<• 4b0 1 i F(2,l6o) ^ 
5.54, j> < .01 ; F{4,100) = 1>43, £ > #10, respectively). Ijline year olds : 
identified more probl ems of each type (3 .91) than seven year olds (3.29) , . who 
i n turn i dent if i e^more than f iye -yea r ol ds (2 . 1 9 ) Internal- i nconsi stenci es 
were sti 11^1 east likely to be identified {2.66) , and identification of 

.. • • k * " ' ' v ' ; . -.. ' ... /.** - . " < ■ '- 

nonsense words and prior knowledge, violations did; not differ (3.15 arid 3w.ll , 
respectively); - ' •. : ^ : v - t : v ; , ; 

The data included in Tabled clearly indicate improvements .in problem : / 
identification when children -were given feedback and a second opportunity to 
listen to and respond to tfie - passages.-- To test for differential improvement 
as ^function of age or problem type, a third analysis of variance was carried 
out which included trial as a factor, the main effect of triaj and the 
interaction with age^were rellablfe^ F(1^50) - 165, 74, £ < ...0Q1 and f (2, 50) m 
4.23, £ < .05, respectively. All subjects Identified more problems when they 
were given a second attempt to find them. The 5 and 7 year olds showed 
greater, gains than the 9 year olds, whose initial levels of performance, were 
so high that there was less room for improvement. Neither tfie interaction of 



trial with problem type nor the trip.le interaction were reliable. These 
results . demonstrate that even the youngest children were able to re-evaluate 
the passages and detect problems in material that hsd previously seemed 
non-problematic*" - ; v ' 
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Standard. appli cations . The dependent measure in the .above analyses was -the "* 
number of times a ehi } d ;i<|ent1 figdvthe /i^ s . , I 

provides- n^ children perceived pth^;r segments of | 

, the passes ^rbbl emit H.;;^ The. fact that these segments were not. intended I 
to: contain problems ts. i^ what is important is how tha children * 1% 

"^interpreted them and the'i r subsequent evaluation of that i nterpretation. In 
•other words,. the focus on problem detection per se may underestimate 
^J:hl]dren^aM^ 

eyaluatldn, In this section, we will consider evidence of standard use* - 
throughout the entire testing session. ' 

Table 3 shows the mean number of times children applied each of the three " 
; targeted;standards; v f Igures reflect standard" use both in the service of 
problem detection and in the evaluation of nbntarget information. An analysis 
of \variance was carried out, With the dependent variable the total number/ of 
times each^chlld applied each standard/; Reli 

of standard were; revealed, \as was an age by standard interaction. Overall , 
the 5-year-old children -had fewer. ^standard applications than the 7- and % 
9-year%qld children who did not differ, £(2,50) - 20.69, j; < ,001. The 
internal consistency standard was applied less frequently than the lexical. 
standard which in turn was applied less frequent Jy^bjtethe external,' 

i consi stency standard , £(2^1 00 ) = 10.08, £ < .001 (fU^lr's Isd 44).. , 
Interpretation of the main effects is qualified by the/1nteract1on, f (4*100). *= , 
2.84, £ < .OS, Although the 7 year olds did not differ from the 9 year olds^ 
in over al 1« standard use, they used the i nternal consi stency standa rd 1 ess 

often and the externaT consistency standard more often^ They^ere actually 
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, c of use of the Toxical - 

standard ;aVso contributing to th# interaction was the fact that the>9 year 
olds&used all three, standards equally often- " 
* : ^ in Table 3 with the Trial 2 data in Table 2 . v ^ ' 

provides Hrifor^ati on about the effectiveness wi t h which children used the : 
. different standards: from an adu lt perspective. Recal 1 that adults 
v consisten^^idehtif led the target problems and rarely identified other 1/ 
^- aspects of the text as probl^at^^ 

^ adults did, with near perfect problem identification and few additional 
problems reported. The younger children r^porfed fewer problems^ but the 
difference ^between the number of times a standard was appl ied and the ^timbent 
of corresponding problems identified was of relatively smalV magnitude. This ^ 
indicates that the children were not simply guessing and identifying anything . 

. v - w ^ -, : ; •,, . •: " . , : \ . • . . ~ 
as; problematic simply to comply with task demands; rather,, they were applying 

-.. • tKe standards selectively, the largest discrepancy between number of problems 



identified and Viumber of times a^standard was applied was show^-^^the^ 7 year 
olds for prior knowledge violations and the external consistency standard. On 
average, ea^n child challenged the truth of 1.63 facts in addition to, the 
targeted facts* But considering that there were a total of 49 sentences", ^each, 
of which contained at least ojii challengeable proposition, this Still reflects 
selectivity in standard use. Nevertheless, the 7 year olds did have a greater 
propensity to challenge the external consistency of passage statements than ~ 
either the younger or older children. - */■ 1 : / . J: 

Additional insight into children's use of ttie different standards can be 
gained by examining patterns of individual use. Are there certain standards 
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tjiat are less likely than other standards to be used at alt? Are younger -"-•'/. 
dhlJI drferr 1 esi 1 1 kel^ ,tp apply ahy^particU tha^^plde^ chi lIlr^liT^ , ^ / W 

To ansWer these questions 9 *iach child transcribed protocol; w^ 
^etferice or absence of each standard, Thi s procedure- revealed thiat all %pdy\ 
9^ear-pld ahildren ;ui tHe;;5^-:V.;^; 
year 61 dV did too. All of the 5 year olds used the- .extern^ 

standard , all butspne used the 1 exlcal standard, and TalV but three used the " , . 
internal. cbnElsttney 

three different- standards. Moreover, if a child used a standard at all he or '■*>■ 
she also usually detetted at 1 east one of the corresponding problems. In only 
two instances did a child use a standard but fail to report any problems (both - . 
children Vrt re ¥ year ol once the" standard v/as the 1 exical i the other t jme^^ ^ 
the standard was the internal consistency). In sum, it : seems safe Jo conclude 
that all three standards were,p l resent in the repertoires of children of all 
ages and that these standards could be used effectively and selectively. 

= • Experiment 2 - y " :■' .\" J j 

Experiment 2 was designed as a replication apd extension of Experiment 1 ^: 
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using an, older jrofip of subjects. The 11-year-old subjects in'thls experiment 
were asked to read the passages on th^r own rather than listen to them, but 
in all other r^^petts the "task was thfe same. The change from listening to t 
reading, which was not expected to affect performance significantly, was made 
to permits a better comparison with studies reputing poor problem detection 
among 10- and li-year^pld children who read passages without being told in 
advance that problems were present (e.g., Garner, 1980* Paris & Myers » 1981). 
After the subjects finished reading and responding to all six passages, they 
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were of pa rti cu Tar i nterest for what the** mi ght + reveal about ? cHi 1 dren 1 $ x v 



Method : ■ r ^ ' . ■-, ' "X-;:.^;..i 

Subjects. The subjects wore 18- children (11 girls) enrolled in the fifth V 

■ 'V::" ; . • ' / 7 ^ ' • y'd 

grade, at the same suburban elementary school as the participants In Experiment 
rlT"The mean~a"geXof :.the~sul)jects r was 11 years, 0 months, 
: '■. Ma teria ls. The materials were the same as those used in Experiment 1. 
Eacch passage was typed double-spaced on a 12.7 x 2d^^^^^.^paji with the , 
title at the top. fc ' \ ' . . 



• V- ^'K -. . . . 



Procedure. The procedure wafs ,sim1 tar ti:i;hatvof Bxperirhferit f l. the 

: ; ? . . . ; : ^ , ' ■^■^r • .• v , , •• 

primary difference was that the subjects read the passages on thei r bwn 

. ; • • \-- ^ . • •• ^ -" ! • \ • ; ■ ^ "■- : . 

i nstead of 1 ^teni ng to them* -To make the task more cpmparab Ve to the / 
Jisi^ning:ta^k, subjects covered up each line of text with a> blank note card 
after they^r^ad it. When they finished the ent i re passage , they pi aced -the 
r card down, on top of the passage and reported whatever probl ems" they had '" : 
noticed. They were given immediate feedback; and re-read the passage if they, 
missed any problems. They were encouraged to report the problems 'as soon as 
• the ^ found them on this^ second trial . If they still failed to report a 
-problem, the experimenter identified it for them. : .,"..-.\ : . . 

After the subjects finished reading all passages, they were asked whether 
they thought some problem types were easier/harder to not i ce than others . 
Half of theVsubjects were asked the. question with the word easie r, half with 
the word harder. If they gave an affirmative answer, they were asked to 
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explain it. They 



were then asked the alternative form of the question 
(harder/easier) and were-aga^n asked to explain. 1 
All sessions were taped and later transcribed. The scoring of the , 
responses and the.#eanse^b6<l protocols was carried out in the Same manner as 
/in Experiment 1 .? . j ■■• • * : ; ; • . y. 

Results and Discussion •, ' •.;;_/ ' ■ ' y.- •.'•* ". % _ : : 

perceiyed problem dlff^ 

the evaluation, of nbn- target as wel l as' target probl ems'. V : X ^\^T 

the number of $rbbl ems: identified »;w1thtri al and probl em type as wit hi n^ 1 
subjects factors. The 

>,0G1 * with fewer Inconsistencies: identified than eftner nonsense words or . ^ 
. prior knowledge-viola 
^rial was reliable- 

ll^year-ol 3 children also benefitted from a second opportunity to evaluate the 
passagesv;;The extent of the Improvement varied according to probl 
evidenced tji the trial by problem type interaction, F{ 2, 30) * 10.88, ;£ < .001. 
0n *.rt*] [-;•?:.• .^^Wl were better identified than prior knowledge" *r .'; 
violations, which in turn were better identified than inconsistencies ' ; 
(Fisher's J5d m .46, £ < .05). On Trial 2, the three problem types Were 
equally well detected, with the gains /from- Trial 1 to Trial 2 "reliable for the 
inconsistencies and prior knowledge Violations . The Tack of a slgniVieant 
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difference for the nonsense words reflects the fact that subjects were near ' 
ceiling on Trial 1, leaving- little room for improvement on Trial 2/ There " 
JV^re no other rel|able>main effects or interactions.":''-^ 
^ Perceived problem difficulty After Wv*WH t>*£k,. 'MMitM 
were, asked .which, „ an,. prohle* typos won, easier to detect and which were 
Harder to 'detect. m ,. h ,. m snbjects. ail but two l^lceied that the nonsense 
words/were the ; eaHe^Tbe labels they used varied, from "nonsense words" to 
weird words" to "misspelled woads" to "words that aren't honest to gosh 
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words that ybd iHild-iverv dav '"^h^**'';.^*'^-'^^"'^ A I V 

.enuonod.-'basebari.b.t^a^lest (a prior knowledge v^m ion), and the • 
other said that "most wore easier to find than the one about checkers" (an . . 
interriaT inconsistency) ".; . : \ - f 0; 

In response to the ••harder" question, 11 children reported that the ' 
inconsistencies wer^har^ 

them harder, Two of these chi } ldr e n^ttribut e d thel difficulty |^ V '--;'; 

Other; you n^d to try to reme.bor both." The third identified a particular 
reading strategy as- the s^rce^her difficulty: "Sometimes whe^ read 1 
skip over words , .1 i ke bi g» smal "s o. I didn't pick them up." of the seven : 
children Who did, riot report inconsistencies as the most difficult, two said " 
prior knowledge violations were th,e hardest and five said «n Q t^ ' ' ' 
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in suni, the reports of perceived difficulty parallel the actual : ^ 
difficulty. However, the reports were probably influenced by prior 
; /perfomancer, slric^tHe ehildrifn had received feedback. In Tact, usually the : 

:M /Vfpifid ^ b^irifdrmati^e^ to ask for; ohil dren 1 s judgments before they carry oiit 
kucb an evaltfatton task. ' - •.' , /4 : i -;.^V« ; ,, ■<?'. " 1 

Standard applications. Tahle^a fhrli^riec the mean number of times each 
child applied the three standard^ of evaluation regardless of whether the 
^sppnsetwfs to an 

standard type as a wi things ubjects factor, reveal ed no dif^erehce ifi the^^^^^ ^ 
^refliMcy ,wi^ 

parallels the lack of a difference in. the identification of the three types of 
prpbl ems by the second trial . The simll arity i^ the means for probl em 
identificatidh and standard, application indicates that the 11 year olds used 
the standards very selectively, in a manned comparable to the atiults. They 
never identified any words "as problematic that were not. intentionally so and 
only twice did sublets identify perceived viol atloris of prior knowledge that 
were not deliberately introduced. Additionally, the Internal consistency 
standard was applied to nonttfrge occasions. 
Moreover, as wduld be expected given the patterns of standard, use among the ■ ; .'= 
older children in Experiment 1 , there were no subjects who failed to use any 
Of the target standards,, nor were there any who fai led to report any type of 
problem. ' '•' --^ ■• •. • . ' . ' .' 

*\ General Discussion " 

(The present experiments have $hown that, acrgss-a wide ag^ r^nge; children 
have considerable ski 1 1 at usi ng several different: standards to evaluate their 



understanding , - Al t frbugh ol der ch t Id fen we re* more^ successful than: youhger^ 4 
chi Itlren , :the Qabsol ute levfel s of ' pert orma nee Were much hi gber than has 
typical ly beeri reported' i n the 1 i tp r^ture* Th£re"are several important v ; 
reasorls for thi s difference: (T) The childreh were £xpl icitly i nfbrmed that 
problems would be present and they were giyenr several- examples of each type. 
Failure to provide this information in advance can lead, to serious under- • 
estimation of comprehension monitoring skills. (Baker, 1979; Stein & Trabasso, 
1982; Win og fad & Johnston, 1981 )^ C2) The -chi-ldren were given immediate 
feedback after each attempt to find a problem. This feedback in essence 
served as instruction in evaluation. (3) The children were asked to 
re^eval uate . the passages for missed, probl ems, gi ving them ah^ opportunity to * - 
improve their performance, (4) Three different types of ^problems vtefe used, 
in recpgnttiori of the fact that comprehension monitoring requires ,'CTe use of 
multiple standards, some of which may be easier to apply than others. 

Of the three standard^ selected for study, the internal consistency 
standard was expected to be most difficult; to apply and the data supported 
this expectation, Children rangl/ig in age: from 5 to 11 were less likely to 
identify internal inconsistencies in the passages than they were the other 
problem types. Evaluating text for Internal consistency Is a cognitively 
demanding task because 1^ requires that the reader or listener first Integrate 
the relevant text propositions. If the propositions are not adjacent, they 
may not be simultaneously active in working memory and so a long term memory \- 
representation must be accessed (Kintsch & 'van Dljk* 1978), The likelihood v 
that adults will carry out this integration during reading decreases with L 
increasing distance ^between the propositions (Walker & Meyer, 1980) * ' , j 



Moreover^ 

; t0 integrate wi-dely Separated propositions even when the necessary premises 
"are ayailaMe in memory, in view of this, evidence, it is hardly surprising 
that. the internal consistency standard is less frequently applied., * ^ 

• because i^ 

po^rditecHolfi-^n^ 



reflected Jn unwi 11^ 

seemingly simple words rather than ineffective use of a lexical standard. The 

^P^e^Maj^ 

olds had near perfect: rtonsense word detectl on, and the 7 y^ar olds were also 
very successful. The one surpnis 

identified no morn nonsense words than internal inconsistencies. However, 
.when given feedback tha 

identification rate increased from 25% to 58%. 

No specific predictions were made regarding the external consistency • 

standard, though there is reason to suspect it would be of intermediate " 
difficulty to use. Since evaluation of external consistency requires - J 
consideration of how ideas in the text relate to what one a 1 ready knows. ' 
greyer cognitive effort is probably required to detect a prior knowledge ' ! 
violation than a nonsense word. And, on the assumption that the relevant ' 
prior knowledge H probably more 

previously encountered text propositions, pAior knowledge violations should be 
more detectable than internal, inconsistencies^ The data do not provide clear 



yidlatjpns than nonsense words, while the 11 yean olds were initially better 
• at 1 dent i tying nonsense words than pri or know! edge viol at i oris , By the second ; 
'trial , however, there were no differences for any chi Idren in detection of 
; nonsense words and* prior knowledge violations. Interpretation of this finding. , 
is difficult because overall detection levels were so high. If the detection 
v rates had been lower, perhaps there would have been "room" for differences to 
/ manifest themselves Increasing the difficulty of the materials could have 
. such an effect , but i t mi ght obscure differences among, t^^^ younger chi Id ren . .:_ 
The risk of encountering either ceiling or floor effects is a familiar one to 
researchers who wish to study developmental change across a wide age range. 
However, since the primary goal in this study was to show what children can do 
under optima V circumstances^ ceiling effects were regarded as a necessary '' 

. eVtl , J " : ■ - ' - \ ' 

... Finally,. the results extend those of Markman and Go r in (1981) in showing * 
& that not only can children adjust their stahdards of evaluation, they can keep 
• several in mind simultaneously. There was, however, a very striking • 
difference; in the performance levels of the children in the two studies, 
; despite the apparent si milarity of materials and task' demands. Even when 
■ .specificalTy instructed to look for a particular type of problem, the 8- and 
10-year-bld Children in Markman and Gorin's study had much lower detection 
rates* than the, phi Idren -of comparable ages in the present study. TKe ./ 
difference may well lie in the feedback that was, provided after each attempt 
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Oyeral V, the resiil ts ja^e 6ncour;a^ i rig in suggest ;i i&j^ 
;indMced-to monitor their po^rehens^ 

intervention. '.'/•'.•• ■■'sT". 1 *- r "" V ' ; y 
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Footnotes 



° • o Four additional five year olds served, as subjects but they were dropped 5t 

. ... %4 1 - ' 

from the study because thei r data records were incomplete due to a 



tape-recorder mat fu^ 

*f ideht 1 f i ed wa^s aval lab! e f rdni the written records made at the time of 

but i nformation about all other i nstances of'standard use depended on 
* 4 transcrfptiohs b 



0ng 5 °l d made a comment /reflecting evaluation of informational 
y cpmpl eteness 7v^ e ^°W s us f the proposi ti^phal cohes/lyeness staywIa^dV 

In addition, 14 5 year olds made one or more nonevaluative comments about the 
passages. \. : ; v ? ;.:<^> 

3 Prior to carrying out a full analysis; of th^ data, the cOTparabrjity of 
the two different Tsamples^^o 

variance was carried out on the data provided by these chi lid ren alone,, wt 
school as a between-subjects factor. There was no main effect of school 
(F(I,20) = 1.19) nor were there any InteracWons of school with the factors of 
interest. ^Accordingly, the da^ 

subsequent analysis. A second prelimi nary analysis, using the full data set, 
included gender as a factor. ^The main; effect was not reliable nor were any " • 
r interactions with genders ; The data were therefore collapsed over gender for 
ease of exposition. tX ';','. 

Use of non-target standards was Infrequent among 11 year, oids as well. "', ,. 
Two students cpmmlrited on the structural cohesiyeness of the passages and one ' 
on the informational completeness* ' r ; ' ! .. • - * ; , 



• ■ 'K'Zfs ■}■ . v .;. : '; ; ;- ; -;: ; - Table l . ' < ; v\\V'0 ^ 

/ - Examples of Passages with Embedded Problems " 

;-:\"^-Sr^"f5-.-4 • : ';• ' 1 ■■. Albert the Pretty Rabbit _ < . v . :' ; v . 

Once there; Was a rabbit named Albert/ He had darkbrown fur that was as soft 
as could be/ 1 He was very 

Jack's Life • 

^ aS - a11 and W b ^ morning Jack chopped 

his f^ 

jquickly on schpolVdaysVi 

He wanted to be a teacher when he grew up. , ; 



- sentences contain an internal inconsistency 
b - sentence contains a. prior knowledge violation 
c 7 sentence contains a nonsense word 
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Table 2 

Mean Number of Problems Identified 



Experiment 1 



Age Trial 



-1 



Nonsense 
Word 

1.00 



2 2.30 



Type of Problem 
Prior 
. Knowledge 
Violation 

2.48 



Internal 
Inconsistency 



0*96 

1*78: 



1 2.38 

2 3.63 



2.62 
3.31 



1.50 
2.94 



Experiment 2 



IT 



1 3.43 

2 4.00 

1 3.7/ 

2 4.00 



3.36 
3.93 

3.28 
4.00 



3.00 
3.79 

2.55 
3.77 




Experiment 1 
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Table 2 _ *V 






Mean Number oF Times Chi 1 dren 


- V" - ; 7 ' 




• A: ; Appl ied; Each Standard : 






Type of Standard ; 


«- •' V-/- '.. \,.V- -". 




C ^ External . 


Internal V 


Age 


Lexical Consistency 


. Consistency ^ 


5 ; ; 


v ;.y_ Mlo-^.^iimy^.:Ji^:' x. 


2.17. • ; ' -;r^; 


7 


4.38 . 4.94 ' 


3.25 ' \ " - 


9 


■v ' : ,', 4.07 • . 4.14 ', \ 


4.21 ' ' ' . ; . : 



Experiment 2 ^ , • ^ 
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